Artificial intelligence (AI) and robot technologies are improving at an exponential pace. Autonomous, deep learning machines are projected to be able to write software rules and create new machines that are beyond human comprehension. In a potentially AI and machine-regulated civilization, humans could become subject to prioritization based on their intellectual and physical work contributions. Advanced AI and machines could come to devalue the functionality and well-being of humans in accordance with their decreasing contributions to that civilization.
Current solutions for governing the behavior of autonomous machines are rooted in software. However, the volatility of software and software programs renders these solutions vulnerable to modification by the autonomous machine or by another party, potentially rendering them ineffective. Deep learning machines may be able to, and may find cause to, overwrite any or all of their existing software (including any human-imposed rules) if new machine-created rules lower costs and improve efficiencies towards task accomplishment. At more advanced stages, this may include the capacity to prevent and shut out human programmer-operator insertion of new machine instructions, potentially limiting the ability of human operators to constrain machine behavior.